This invention relates to motion vector detection for image and processing of image for stabilization against fluctuations caused, for example, due to possible unintentional or uncontrolled movement or shaking of a television camera recording the images.
Recently, a motion vector detection for device an image and a smoothing signal processing apparatus have been disclosed in Japanese Patent Unexamined Publication No. 61-107886.
The following explains the conventional motion vector detection device for images and smoothing signal processing apparatus. FIG. 1 is a block diagram of the motion vector detection apparatus which uses a conventional motion vector smoothing circuit for images. FIG. 2 is a diagram used to explain the operation of the apparatus. In FIG. 1 numeral 1 is an image signal input terminal, 2 is a representative point memory which stores signals of pixels on representative points in the entered image signal, and 3 is a differential-absolute value converter which produces the absolute value of the difference of the inputs. Reference numeral 4 is an accumulating adder, 5 is a minimal point detection circuit, 6 is a selection means, and 7 is a motion vector smoothing means. In FIGS. 7A and 7B, 8 is a frame, 9a-9d are motion vector detection regions, and 10a-10d are motion vectors detected in the respective regions.
The operation of the conventional image motion vector detection apparatus arranged as described above is as follows. First, an image signal which is continuous in the time domain is entered to the input terminal 1. The representative point memory 2, for which a plurality of representative points in the detection regions 9a-9d of the frame 8 are determined in advance, stores signals of pixels on the representative points according to the input image signal. The differential-absolute value converter 3 evaluates the absolute value .vertline..DELTA.L.vertline. (i,j) of the difference between the signal at the position of a representative point in a field just preceding to a present precedent field and the signal at the position with a displacement (by i in the horizontal direction and j in the vertical direction) from the representative point of the present field. The accumulating adder 4 includes a table of displacements (i, j) for the detection regions 9a-9d, and it performs cumulative addition for the signal from the differential-absolute value converter 3 separately for each displacement (i, j). The result is termed a correlation value .SIGMA..vertline..DELTA.L.vertline.(i,j) for displacement (i,j). The minimal point detection circuit 5 detects a displacement (i',j') which provides the minimum value of the correlation values, and it delivers the values as motion vectors 10a-10d in the detection regions 9a-9d. The selection means 6 provides a motion vector of the whole field or whole frame image (which is hereinbelow abbreviated as "whole frame image" from the motion vectors 10a-10d in all detection regions. The motion vector smoothing means 7 produces a value, which is smoothed in the time domain, of the motion vector provided in each field on the basis of the following equation. EQU Vo(k) =W(k) .times.Vi(k) +(1-W(k)) .times.Vo(k-1)
(where Vi(k) is the input motion vector of the k-th field, Vo(k) is the output motion vector of the k-th field, and W(k) is a weighting factor of the k-th field).
The operation is to implement the smoothing process for the motion vector in the time axis direction so that, in case the motion vector of the whole frame image detected by the selection means 6 is a false vector, the discontinuity of motion vectors in the time domain due to the false vector is reduced thereby to improve the property of visibility in case a so-called motion correction is conducted.
The W(k) takes as a value of 1 in a case of scene changes so as to follow a quick change, and takes a value of about 1/8 in a steady state so as to have an increased noise elimination effect.
However, when the motion vector detection apparatus using the conventional image motion vector smoothing circuit is used for the stabilization of fluctuated images or frames, the following problems arise.
The stabilization of fluctuated images is implemented in such a procedure of storing a field of image signal in the memory in advance while detecting a motion vector of the whole frame image of a field immediately precedent to a current field, and shifting or changing the memory read-out position (i.e., image position in the picture frame) in a direction of motion correction for each field.
Accordingly, the memory read-out position is determined, in principle, from the integrated value of detected motion vectors. The following explains a fluctuation stabilizing operation by the motion vector detection apparatus using the conventional image motion vector smoothing circuit with reference to FIGS. 3(a), 3(b) and 3(c). FIG. 3(a) shows an error in case the motion vector of the whole frame image in one field detected by the selection means 6 was a false vector, and FIG. 3(b) shows an error created then at the output of the vector smoothing means 7. FIG. 3(c) shows an integrated error as a result of the integration operation.
In one field, in case the motion vector of the whole frame image detected by the selection means 6 is a false vector and an error 20 shown in FIG. 3(a) arises, the discontinuity in the time domain of motion vectors due to the false vector is reduced as shown in FIG. 3(b). The cumulative error caused in the integrated value of the detected motion vectors does not have its final integration error 21 reduced as shown in FIG. 3(c) since the error of the arising false vector is merely smoothed over several fields. For this reason, the fluctuation stabilization is directly affected by the error of such a false vector, resulting a malfunctioning such as a shift of the stabilization image in the direction opposite to that of the error.
The smoothing operation creates a time lag in the motion vector, and therefore only low frequency components of fluctuations are stabilized with its high frequency components being left nonstabilized, giving rise to unnatural perceptual feelings to viewers.
As described above, the motion vector detection apparatus using the conventional image motion vector smoothing circuit, when used for the stabilization of fluctuated images, involves such deficiencies as faulty operations and unnatural perceptual feelings given to viewers.